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In recent years, discussion has arisen among 
educational researchers and practitioners on how best to 
teach academicaliy underprepared community coiiege 
students the basic skiiis they need to be able to learn from 
a college curriculum. Research conducted by the 
Community College Research Center (Perin & Charron, 
2006) has found that many creative approaches are used 
in developmental education, but there exists little 
quantitative evidence on their effectiveness. The study 
summarized here begins to fill this research gap. With 
funding from the U.S. Department of Education’s Institute 
of Education Sciences, an intervention called the Content 
Comprehension Strategy Intervention (CCSI) was 
developed and tested at three community colleges. CCRC 
researchers drafted and pilot-tested the intervention in 
collaboration with science and developmental education 
faculty and senior administrators at Bronx Community 
College. The intervention was further tested and revised at 
Los Angeles Pierce College and Norwalk Community 
College. In this Brief, we describe the intervention and 
present data suggesting that it is a promising strategy for 
community college students who need to improve their 
reading and writing skills. 

The Intervention 

The CCSI is a curricular supplement for upper-level 
developmental reading and writing courses, i.e., courses 
one level below college English. The aim of the 
intervention is to provide systematic practice in basic 
academic skills to augment and strengthen the learning 
occurring in developmental education classrooms. Self- 
directed and self-paced, the CCSI is used independently 
by students outside of class. It consists of 1 0 separate 
units, each taking 1-2 hours to complete (we developed 
two versions, each with 10 units, described below). The 
course instructor assigns one unit per week during one 
college semester. Students complete the work on their 
own time and return the completed unit to the instructor. 
The CCSI is also appropriate for use with tutors in a 
college academic learning center to help students develop 
reading and writing skills needed in disciplinary courses. 

The ability to understand printed text and write 
analytically is critically important to academic learning, 
especially in college. Research on the cognitive 
psychology of literacy points to several skills that promote 



reading comprehension: the ability to summarize 
information in writing, to ask questions of oneself while 
reading, and to apply vocabulary knowledge (Edmonds et 
al., 2009; Gil, Braten, Vidal-Abarca, & Stromso, 2010; 
Rosenshine, Meister, & Chapman, 1996). Another 
important area is persuasive writing, which is often 
assigned but can be a difficult task for students even at 
the postsecondary level (Nussbaum & Schraw, 2007). 
Practice in these skills forms the core of the CCSI. In 
addition, students receiving the intervention practice 
answering multiple-choice reading comprehension 
questions similar in format to those found on high-stakes 
college reading tests. 

Each week, the students participating in the study 
used the CCSI to practice the same skills — written 
summarization, question formulation, defining and using 
vocabulary, persuasive writing, and reading test 
preparation — using different reading passages. Varying 
levels of support, depending on the skill being practiced, 
were provided within the intervention in the form of 
guidelines. The most heavily supported skill was written 
summarization. Many college assignments, ranging from 
research papers and laboratory reports to personal 
memoir, require that information be summarized. To write a 
summary, it is necessary to distinguish between important 
and less important ideas, which underprepared college 
students can find difficult (Caverly, Nicholson, & Radcliffe, 
2004). To support students as they prepared to write 
summaries in the CCSI, we provided highly focused 
questions that drew attention to the key ideas in a given 
reading passage. The students then used their answers to 
these “main idea” questions to write the summary. Over 
time, this support was reduced in order to help students 
summarize more independently. The instructions also 
indicated that in writing a summary, the students should 
paraphrase rather than copy from the reading passage, 
since earlier research had found a great deal of word-for- 
word copying in the summarization of developmental 
education students (Perin, Keselman, & Monopoli, 2003). 

Minor support was also provided to help students 
comprehend the reading passages and represent 
information accurately in their summaries through a self- 
monitoring checklist that was completed by students after 
they had read the material in the unit. No support was 
provided for using the conventions of English (mechanics, 
such as proper grammar, punctuation, and spelling) while 
writing or for the other components of the intervention; 
rather, the students simply practiced using correct English 
through the act of writing. 

Contextualization 

An important feature of the intervention is that it is 
contextualized. Contextualization is an approach in which 
skills are taught with direct reference to real-life situations 
in order to make the skills meaningful to students 
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(Johnson, 2002). Instead of teaching reading and writing 
skills in the abstract, contextualized literacy instruction 
focuses on “authentic content.” The guiding assumption is 
that skills learned through contextualized instruction are 
more likely to transfer beyond the course in which they are 
taught. According to this theory, such transfer would result 
from the similarity between the contexts of learning and 
eventual application as well as from increased student 
motivation. 

Two sets of intervention units were developed, one 
contextualized and one not. The first set was 
contextualized in biology, authentic content in that many 
community college students must pass a science course 
to earn a degree. We selected biology because a large 
number of students, including many who aspire to health 
careers, fail this subject, in part because they have 
difficulty with the reading requirements. The biology units 
were anchored in five topics, all from anatomy and 
physiology: matter and energy, atoms, the heart, blood, 
and respiratory system functions. Each topic was used for 
a set of two units: the first unit in each pair used easier 
text in order to help students develop background 
knowledge, and the second used introductory college- 
level text on the same topic. 

The other set of intervention units used a variety of 
unrelated themes drawn from developmental education 
textbooks. This set of units used 10 different topics, all on 
high-interest issues: genetic testing, entrepreneurship, 
censorship, drug addiction, the social consequences of air 
conditioning, the social role of news media, cosmetic 
surgery, the participation of African Americans in baseball, 
youth hazing, and the founding of Liberia. 

Each student used either the biology or the 
developmental education version of the CCSI throughout 
the semester. Below, we refer to the biology text as 
“science” and the developmental education text as 
“generic.” Both sets of units provided practice in the same 
reading and writing skills and were formatted in exactly the 
same way; what differed was the subject matter that 
formed the backdrop of the practice. Thus, every unit, 
regardless of subject matter, required that students 
practice summarizing a reading passage, formulating 
questions, working with vocabulary, writing a persuasive 
essay, and answering test-prep questions. While the 
intervention was being developed, Richard Bailey, faculty 
at Henry Ford Community College, worked with the project 
staff to conduct telephone interviews with a purposive 
sample of participants to determine their interpretation of 
intervention tasks. This information was taken into account 
during revision of the intervention. 

Data and iVlethods 

Sample 

Six cohorts of community college developmental 
education students took part in the intervention during the 
project, which, as mentioned above, involved developing, 
testing, and retesting several iterations of the intervention. 
This Brief reports the findings for the final and fully 
developed version, which we administered to the last 
cohort, comprised of students attending 16 upper-level 
developmental reading and English courses at Los 
Angeles Pierce College and Norwalk Community College. 
The cohort consisted of 369 students, but we confined our 
data analysis to the 246 students who completed both the 
pre- and post-tests. Thirty-four percent of the students in 
the sample were of Hispanic background, 21 percent were 



White, and 20 percent were Black. Fifty-five percent were 
female, 60 percent were full-time students, 57 percent 
were aged 1 9 years or younger, and 67 percent had not 
taken prior remedial courses. 

Measures 

Two instruments were employed to assess the amount 
of gain associated with the intervention. The first, 
developed specifically for the project, was the Science 
Summarization Test, which presented a task closely 
related to the summarization practice received in the CCSI. 
During the test, however, the task is presented without the 
support provided within the intervention in order to assess 
students’ ability to write a summary as in a typical college 
assignment. Alternate forms A and B of the instrument 
were developed and counterbalanced at pre- and post- 
test points to control for text effects. 

We scored the students’ writing samples on five 
dimensions: the number of main ideas from the source text 
that appeared in the summary, the accuracy of information 
in the summary, word count, the extent to which students 
paraphrased rather than copied the source text, and the 
use of writing conventions. A Teachers College faculty 
member, Stephen Peverly, worked with the project team to 
identify the main ideas in the source text. The number of 
main ideas in each student’s summary was expressed as a 
proportion of the number of main ideas in the source text. 
Accuracy and conventions were measured on a four-point 
scale, and paraphrasing was scored on a two-point scale. 
Inter-rater reliabilities for the proportion of main ideas on 
the alternate forms were high (r = .91 ; r = .94). 

Serious academic difficulty was apparent in the writing 
samples even though the participants were in the upper 
level of developmental education. Sentence structure was 
often poor, making it difficult for scorers to determine 
where sentences began and ended and what the writer 
had intended to say. Consequently, inter-scorer 
agreements were somewhat low (accuracy, 52 and 66 
percent; conventions, 45 and 57 percent; paraphrasing, 83 
and 79 percent; in each case for Form A and for Form B, 
respectively). 

The second instrument was the Nelson-Denny 
Reading Test (Brown, Fishco, & Hanna, 1993), which we 
used to assess transfer from the contextualized reading 
skills practiced in the intervention to general reading ability. 
This measure, which is frequently administered to college 
students, is comprised of two subtests: vocabulary and 
reading comprehension. The scores for the subtests are 
summed to create a total score, which is then converted 
to a scaled score {M = 200, SD = 25). At the pre-test point 
we also administered two additional experimental 
measures, developed in collaboration with Linda Mason, 
Pennsylvania State University faculty, to assess students’ 
knowledge of and interest in science. These measures 
served as covariates in the study. 

Design 

This was a quasi-experimental study in which faculty 
who taught 16 upper-level developmental reading and 
English course sections were recruited by college 
administrators. Twelve of the 1 6 course sections were 
assigned by the college to participate in the intervention 
(treatment group) and the other four were recruited to take 
the pre- and post-tests but not participate in the 
intervention (business-as-usual comparison group). Within 
each treatment course section students were randomly 
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assigned by CCRC project staff to use the science or 
generic version of the intervention. The data reported 
below are for students who completed both pre- and post- 
tests. 

Findings 

The Intervention was found to be effective on several 
variables. An Initial series of within-subject f-tests 
Indicated that both the science and generic groups 
showed statistically significant gain in the proportion of 
main ideas identified on the Science Summarization Test, 
while the comparison group showed no significant 
difference on this measure. The science group also 
showed a significant increase in accuracy and word count 
from pre- to post-test, but the amount of paraphrasing 
decreased. The generic and comparison groups showed 
significant pre-post gain on the Nelson-Denny Test, but the 
science group did not. Neither the generic nor the 
comparison group showed statistically significant pre-post 
differences in accuracy, word count, or paraphrasing. 

None of the groups showed statistically significant gain in 
the conventions of writing. 

To compare groups on the amount of pre-post gain 
associated with the use of the CCSI, the post-scores on 
the Science Summarization Test (the main outcome 
measure) and the Nelson-Denny Test (the transfer 
measure) were compared between groups using an 
analysis of covariance (ANCOVA) that controlled for pre- 
test scores. This method allowed us to account for any 
pre-existing differences between the treatment and 
comparison groups. Step 1 of each model adjusted for all 
background variables (science knowledge, science 
Interest, and student characteristics) found in pre- 
screening to be related to the dependent variable, site of 
data collection, and pre-test score. Step 2 introduced 
group status (1 = intervention; 0 = comparison) to 
determine whether the post-test scores varied by group, 
controlling for the scores used in Step 1 . The regression 
weights are measures of effect size in predicting 
standardized post-test scores from intervention, 
standardized pre-test scores, and background 
characteristics. The regression weights for the binary 
group membership variable show the post-test 



standardized scores between the two groups, controlling 
for pre-test standardized scores. Separate analyses were 
conducted to compare the contextualized science 
condition to the business-as-usual comparison, the 
developmental education generic condition to the 
comparison group, and the science to the generic group. 
The unadjusted pre- and post-scores are shown in Table 
1 . All regression analyses for the Science Summarization 
Test used z-scores (M = 0, SD = 1) to provide a common 
metric for interpretation. 

On the Science Summarization Test, both the science 
and generic text groups showed large statistically 
significant gain compared to the business-as-usual group 
on the proportion of main ideas that appeared in the 
written summary (ES = 0.62, p < .01 ; and ES = 0.36, 
p < .05 respectively). The difference was also statistically 
significant between the two intervention groups, with the 
science contextualization group showing greater gain than 
the generic text group (ES = 0.32, p < .05). The summaries 
written by the science group were more accurate than 
those of the comparison group (ES = 0.44, p < .05), but 
the generic group did not gain more than the comparison 
group. The science group’s summaries were also more 
accurate than those of the generic group (ES = 0.33, 
p < .05). Students in both the science and generic 
conditions wrote more than the comparison group 
(ES = 0.70, p < .001 ; ES = 0.62, p < .001). However, the 
gain in word count did not differ between the science and 
generic groups. On the conventions measure, the gain was 
similar for the science and comparison groups. The gain 
on conventions was significantly lower in the generic than 
the comparison group (ES = -0.39, p < .05), and the 
science and generic groups did not differ in the amount of 
gain they showed on the conventions measure. 

Since the paraphrasing measure on the Science 
Summarization Test was dichotomous, logistic regression 
was performed, again using the post-score as the 
outcome variable, controlling for pre-score (all scores were 
transformed to z-scores). The comparison group showed 
greater gain on paraphrasing than did the science group. 
The students in the comparison group were, on average, 
four times more likely to use their own words rather than 
copy word for word, controlling for all of the other 
variables in the model. 



Table 1. Unadjusted Means and Standard Deviations for Pre-Test and Post-Test Scores 



Assessment Scores 



Measure 


Total Sample 


Science Group 


Generic Group 


Comparison Group 


Pre-Test 


Post-Test 


Pre-Test 


Post-Test 


Pre-Test 


Post-Test 


Pre-Test 


Post-Test 


M 


SD 


M 


SD 


M 


SD 


M 


SD 


M 


SD 


M 


SD 


M 


SD 


M 


SD 


Nelson-Denny Reading Test 


































Total Scale Score 


181.11 


21.22 


185.87 


22.36 


183.19 


20.89 


185.81 


21.31 


182.09 


21.55 


188.19 


21.85 


176.06 


21.11 


181.96 


24.83 


Science Summarization Test 


































Proportion of Main Ideas 


0.39 


0.18 


0.46 


0.23 


0.41 


0.19 


0.52 


0.22 


0.40 


0.19 


0.48 


0.22 


0.33 


0.15 


0.32 


0.21 


Word Count 


106.05 


35.62 


108.60 


41.65 


108.21 


34.22 


117.43 


39.72 


109.57 


38.17 


115.03 


35.62 


95.40 


31.85 


79.84 


43.78 


Accuracy 


2.90 


0.66 


3.06 


0.63 


2.99 


0.68 


3.22 


0.63 


2.90 


0.61 


3.01 


0.54 


2.74 


0.69 


2.86 


0.71 


Paraphrasing 


0.83 


0.38 


0.69 


0.46 


0.78 


0.42 


0.57 


0.50 


0.83 


0.38 


0.74 


0.44 


0.91 


0.29 


0.84 


0.37 


Conventions 


2.86 


0.83 


2.82 


0.84 


2.77 


0.88 


2.77 


0.79 


2.85 


0.81 


2.66 


0.84 


3.07 


0.71 


3.22 


0.82 


Science Covariates 


































Science Knowledge Test 


10.82 


2.90 


- 


- 


10.95 


3.18 


- 


- 


10.85 


2.49 


- 


- 


10.45 


3.12 


- 


- 


Science Interest Inventory 


27.06 


5.45 


- 


- 


27.41 


5.44 


- 


- 


27.31 


5.30 


- 


- 


25.87 


5.74 


- 


- 



Notes: The Nelson-Denny Reading Test is a standardized measure {M = 200; SD = 25). The proportion of main ideas scores in the Science Summarization Test are in proportion form 
counterbalanced. Maximum values for Science Summarization Test variables: accuracy = 4; conventions = 4; paraphrasing = 1 . Maximum values for science covariate variables: 
Science Knowledge Test = 20; Science Interest Inventory = 40. Sample sizes vary based on group and assessment measure. Nelson-Denny Reading Test [n = 219); science group 
{n = 85); generic group (n = 85); comparison group (n = 49). Science Summarization Test {n = 199); science group {n = 82); generic group; (n = 77); comparison group (n = 40). Science 
Knowledge Test [n = 234); science group (n = 97); generic group (n = 97); comparison group (n = 40). Science Interest Inventory {n = 236); science group [n = 94); generic group 
{n = 94); comparison group (n = 48). 
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Also, it was found that experimental condition 
(science, generic, comparison) was not a statisticaiiy 
significant predictor of post-test Nelson-Denny totai scaied 
scores in any of the anaiyses, controliing for pre-score and 
the other pre-screened variables. 

Conclusion 

The Content Comprehension Strategy Intervention, a 
curricuiar supplement for upper-level developmental 
reading and writing students in community colleges, 
showed a promising pattern of results. Our results are 
particularly encouraging in view of the lack of previous 
studies with evidence on effectiveness, i.e., studies that 
compare pre- and post-scores for basic skills interventions 
using a comparison group. The CCSI provided practice in 
several areas of literacy that enable learning in college: 
summarizing dense, expository text; formulating questions 
while reading; learning vocabulary; writing persuasive 
essays on controversial topics related to a reading 
passage; and answering traditional multiple-choice 
questions that appear on tests. 

The intervention, whether contextualized in science or 
using generic developmental education text, resulted in 
statistically significant gain on several variables, most 
notably the proportion of main ideas from a source text 
included in a student-written summary. Since identifying 
main ideas in text can be challenging for academically 
underprepared students, this finding represents an 
important improvement. After completing the CCSI, the 
contextualized science group identified 52 percent of the 
main ideas, up from 41 percent, and the generic group 
identified 48 percent of the main ideas, up from 40 
percent. The business-as-usual comparison group did not 
improve on this measure. It is encouraging to note that the 
group practicing basic skills using science text also 
improved in the ability to present accurate information in a 
summary. What is less encouraging is the lack of gain on 
the standardized reading test (Nelson-Denny) and in the 
conventions of writing. Although neither general reading 
skills nor writing conventions were targeted in the 
intervention, gains in these areas might be expected since 
the students were attending developmental education 
courses while using the intervention. 

In any case, the positive pattern of pre-post results on 
measures directly related to the intervention activities, 
especially the inclusion of main ideas and the accuracy of 
written summaries, suggests that the CCSI would be a 
useful addition to developmental education in community 
colleges to help students prepare for college-level 
discipline-area courses that require the reading of highly 
factual, densely written text. One notable finding is that 
both the science contextual ization and the generic text 
conditions showed the same pattern of results for the 
proportion of main ideas identified in a summary, while 
only the science contextualization condition seemed to 
promote the accuracy of information summarized. 

An area for further investigation is the increased 
prevalence of word-for-word copying among the science 
contextualization group relative to the comparison group. 
Table 1 shows that knowledge of science was low in all 
three groups. One hypothesis is that with weekly exposure 



to the science material, students in the science group 
began to realize that they had limited knowledge of the 
anatomy and physiology material and resorted to copying, 
feeling less confident in their ability to express unfamiliar 
concepts in their own words. This and other dimensions of 
the study will be discussed in a longer paper currently in 
preparation. 

A limitation of the study is that the grant project did 
not have the means to administer maintenance or 
additional transfer measures. Future research would be 
required to learn whether the benefits are generalizable 
and long-lasting. 
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